Impact of Data Organization on Distributed Storage System

With the explosive growth of data stored in digital format, there is a need for a new approach to data storage. Large amount of stored data requires modern storage systems to be scalable and easily extendable on-line. Moreover, the data must be resilient and highly available, which in turn requires failure-tolerant and highly available storage. To address these needs a new storage segment called scalable distributed storage systems (DSS) has recently emerged. One of the key architectural decisions for a DSS system is the design of its data organization defining data distribution among multiple nodes as well as local data placement on each node. The fulfillment of almost all the requirements of DSS systems depends directly on proper data organization. In this thesis we describe DSS requirements dependent on data placement. Moreover, we identify trade-offs among these requirements and discuss how tradeoff resolution is related to design choices for data organization. Next, we propose a novel data organization that resolves the trade-offs in a reasonable way. This is verified by using the proposed organization in a commercial DSS system called HYDRAstor. We have a substantial first-hand experience with this system, as for the last few years we have been part of a core team designing and implementing HYDRAstor. In this system, data is organized around a distributed hash table with virtual supernodes spanned over physical nodes. Data resiliency is provided with erasure codes, with fragments of erasure-coded blocks distributed among supernode components. Fragments are stored in containers that are organized into chains to allow fast storage of data streams and enable efficient data consistency management, data health verification and data reconstruction. We conclude that the design of data organization in HYDRAstor allows for the balancing of conflicting DSS requirements which has resulted in the creation of a break-through, highly innovative storage system.